Endoscope system, image processing device, total processing time detection method, and processing device

ABSTRACT

An endoscope system includes: an endoscope configured to generate an imaging signal and output the generated imaging signal; an image processing device configured to perform image processing on the imaging signal input from the endoscope; a display configured to display an image of a subject based on the imaging signal subjected to the image processing by the image processing device; and a first processor configured to calculate a sum of a first processing time from when the endoscope generates the imaging signal to when the endoscope outputs the imaging signal, a second processing time from when the image processing device receives the imaging signal to when the image processing device outputs the imaging signal to the display, and a third processing time from when the display receives the imaging signal to when the display displays the image based on the imaging signal.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of International Application No.PCT/JP2019/008466, filed on Mar. 4, 2019, the entire contents of whichare incorporated herein by reference.

BACKGROUND 1. Technical Field

The present disclosure relates to an endoscope system including anendoscope, an image processing device, a total processing time detectionmethod, and a processing device.

2. Related Art

In the medical field, an endoscope system is used for observing theinside of a subject. In general, an endoscope inserts an elongatedflexible insertion portion into a subject such as a patient, illuminatesillumination light supplied by a light source device from a distal endof the insertion portion, and captures an in-vivo image by receivingreflected light of the illumination light by an imaging portion at thedistal end of the insertion portion. The in-vivo image captured by theimaging portion of the endoscope is subjected to predetermined imageprocessing in a processing device of the endoscope system, and thendisplayed on a display of the endoscope system. A user such as a doctorobserves an organ of a subject on the basis of the in-vivo imagedisplayed on the display.

Meanwhile, in the processing device, a processing time from acquisitionof the in-vivo image to generation of image data for display variesdepending on the type of image and the type of image processing. Whenthe processing time is long, a time lag from when an image is capturedby the endoscope to when the image is displayed on the display device isalso long. When the time lag becomes long, a deviation may occur betweenan actual position of the endoscope in the subject and a position of theendoscope recognized by an operator viewing the display device, and thetreatment to an appropriate position may not be performed.

As for the display time lag, technology for performing signal processingin consideration of a processing delay of an apparatus is known (forexample, see JP 2011-036488 A). In JP 2011-036488 A, an image processingdevice and a processor are synchronized with each other, so that imagesdisplayed by display devices connected to the image processing deviceand the processor are synchronized with each other.

SUMMARY

In some embodiments, an endoscope system includes: an endoscopeconfigured to capture an image of a subject, generate an imaging signal,and output the generated imaging signal; an image processing deviceconfigured to perform image processing on the imaging signal input fromthe endoscope; a display configured to display the image of the subjectbased on the imaging signal subjected to the image processing by theimage processing device; and a first processor including hardware, thefirst processor being provided in any one of the endoscope, the imageprocessing device, and the display, the first processor being configuredto calculate a sum of a first processing time from when the endoscopegenerates the imaging signal to when the endoscope outputs the imagingsignal, a second processing time from when the image processing devicereceives the imaging signal to when the image processing device outputsthe imaging signal to the display, and a third processing time from whenthe display receives the imaging signal to when the display displays theimage based on the imaging signal.

In some embodiments, provided is an image processing device that isconnected to an endoscope and a display, the endoscope being configuredto capture an image of a subject, generate an imaging signal, and outputthe generated imaging signal, the display being configured to displaythe image of the subject. The image processing device includes: aprocessor including hardware, the processor being configured to performimage processing on the imaging signal input from the endoscope,calculate a sum of a first processing time from when the endoscopegenerates the imaging signal to when the endoscope outputs the imagingsignal, a second processing time from when the image processing devicereceives the imaging signal to when the image processing device outputsthe imaging signal to the display, and a third processing time from whenthe display receives the imaging signal to when the display displays theimage based on the imaging signal.

In some embodiments, a total processing time detection method includes:acquiring a first processing time from when an endoscope generates animaging signal to when the endoscope outputs the imaging signal;acquiring a second processing time from when an image processing devicereceives the imaging signal to when the image processing device outputsthe imaging signal to a display; acquiring a third processing time fromwhen the display receives the imaging signal to when the displaydisplays the image based on the imaging signal; and calculating a sum ofthe first processing time, the second processing time, and the thirdprocessing time.

In some embodiments, a processing device includes: an input portionconfigured to receive an imaging signal from an endoscope; an outputportion configured to output a display image to a display; and aprocessor including hardware, the processor being configured to acquirea processing time from when the input portion receives the imagingsignal to when the output portion outputs the display image, and performnotification processing when the processing time is equal to or morethan a predetermined threshold.

In some embodiments, a processing device includes: an image processingcircuit configured to execute first image processing and second imageprocessing on an imaging signal, the second image processing partiallydifferent from the first image processing, a processing time of thesecond image processing being shorter than a processing time of thefirst image processing; and a control circuit configured to acquire aprocessing time from when an image sensor outputs an imaging signal towhen a display displays an image, determine whether the processing timeis equal to or more than a predetermined threshold, and performswitching from the first image processing to the second image processingwhen the processing time is equal to or more than the predeterminedthreshold.

In some embodiments, a processing device includes: a processor includinghardware, the processor being configured to acquire a time from when theimage sensor outputs an imaging signal to when a display displays animage generated based on the imaging signal, and perform notificationwhen the time is equal to or more than a predetermined threshold.

In some embodiments, a processing device includes: a processor includinghardware, the processor being configured to perform image processing onan imaging signal to generate an image, acquire a time from when animage sensor outputs the imaging signal to when a display displays theimage, and switch the image processing from first image processing tosecond image processing when the time is equal to or more than apredetermined threshold, the second image processing being imageprocessing in which a part of the first image processing is thinned out.

The above and other features, advantages and technical and industrialsignificance of this disclosure will be better understood by reading thefollowing detailed description of presently preferred embodiments of thedisclosure, when considered in connection with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a schematic configuration of anendoscope system according to a first embodiment of the disclosure;

FIG. 2 is a block diagram illustrating a schematic configuration of theendoscope system according to the first embodiment of the disclosure;

FIG. 3 is a diagram illustrating a configuration of an image processingportion included in the endoscope system according to the firstembodiment of the disclosure;

FIG. 4 is a diagram illustrating an image displayed by image processingperformed by the endoscope system according to the first embodiment ofthe disclosure;

FIG. 5 is a flowchart illustrating delay detection processing performedby the endoscope system according to the first embodiment of thedisclosure;

FIG. 6 is a flowchart illustrating delay detection processing performedby an endoscope system according to a second embodiment of thedisclosure;

FIG. 7 is a flowchart illustrating delay detection processing performedby an endoscope system according to a third embodiment of thedisclosure;

FIG. 8 is a block diagram illustrating a schematic configuration of anendoscope system according to a fourth embodiment of the disclosure; and

FIG. 9 is a diagram illustrating delay time acquisition processingperformed by the endoscope system according to the fourth embodiment ofthe disclosure.

DETAILED DESCRIPTION

Hereinafter, embodiments of the disclosure will be described. In anembodiment, a medical endoscope system that images and displays asubject such as a patient will be described as an example of anendoscope system according to the disclosure. Further, the disclosure isnot limited by the embodiment. Further, in the description of thedrawings, the same components will be denoted by the same referencenumerals.

First Embodiment

FIG. 1 is a diagram illustrating a schematic configuration of anendoscope system according to a first embodiment of the disclosure. FIG.2 is a block diagram illustrating a schematic configuration of theendoscope system according to the first embodiment.

An endoscope system 1 illustrated in FIGS. 1 and 2 includes an endoscope2 that captures an image of a subject by inserting a distal end portioninto the subject, a processing device 3 that has an illumination portion3 a generating illumination light emitted from the distal end of theendoscope 2, performs predetermined signal processing on an imagingsignal captured by the endoscope 2, and generally controls the entireoperation of the endoscope system 1, and a display device 4 thatdisplays an in-vivo image generated by the signal processing of theprocessing device 3.

The endoscope 2 includes an insertion portion 21 that has a flexibleelongated shape, an operating portion 22 that is connected to the sideof a proximal end of the insertion portion 21 and receives inputs ofvarious operation signals, and a universal cord 23 that extends in adirection different from a direction where the insertion portion 21extends from the operating portion 22 and incorporates various cablesconnected to the processing device 3 (including the illumination portion3 a).

The insertion portion 21 includes a distal end portion 24, a bendingportion 25, and an elongated tube portion 26. The distal end portion 24incorporates an image sensor 244 in which pixels generating a signal byreceiving light and performing photoelectric conversion aretwo-dimensionally arranged. The bending portion 25 is bendable andincludes a plurality of bending pieces. The elongated flexible tubeportion 26 is connected to the side of a proximal end of the bendingportion 25 and has flexibility. The insertion portion 21 is insertedinto a body cavity of the subject and captures an image of the subjectsuch as a living tissue at a position where external light does notreach by the image sensor 244.

The distal end portion 24 has a light guide 241 that is configured usinga glass fiber or the like and forms a light guide path of light emittedby the illumination portion 3 a, an illumination lens 242 that isprovided at a distal end of the light guide 241, a condensing opticalsystem 243, the image sensor 244 that is provided at an image formingposition of the optical system 243, receives light condensed by theoptical system 243, and photoelectrically converts the light into anelectric signal, a correlated double sampling (CDS) portion 245 thatreduces a noise component included in an analog imaging signal using aCDS method, an A/D converter 246 that converts the analog imaging signaloutput through the CDS portion 245 into a digital signal, and a memory247.

The optical system 243 is configured using one or more lenses, and hasan optical zoom function for changing an angle of view and a focusfunction for changing a focal point.

The image sensor 244 photoelectrically converts the light from theoptical system 243 to generate an electric signal (image signal).Specifically, the image sensor 244 has a light receiving portion 244 ain which a plurality of pixels, each of which has a photodiodeaccumulating a charge according to a light amount, a capacitorconverting the charge transferred from the photodiode into a voltagelevel, and the like, are arranged in a matrix, and each pixelphotoelectrically converts the light from the optical system 243 togenerate an electric signal, and a reading portion 244 b thatsequentially reads the electric signal generated by a pixel arbitrarilyset as a reading target among the plurality of pixels of the lightreceiving portion 244 a and outputs the electric signal as an imagingsignal. An imaging signal including image data constituting informationof a display image is output from the image sensor 244. The image sensor244 is realized by using, for example, a charge coupled device (CCD)image sensor or a complementary metal oxide semiconductor (CMOS) imagesensor.

The memory 247 stores an execution program and a control program toexecute various operations by the image sensor 244, identificationinformation of the endoscope 2, and a delay time required for signalprocessing in the endoscope 2. The identification information includesidentification information (ID) of the endoscope 2, a model year,specification information, a transmission method, and the like. Further,the memory 247 may temporarily store image data or the like generated bythe image sensor 244. The delay time corresponds to a delay time of datatransmission caused by the signal processing, and is set by the numberof frames here. Specifically, the delay time is a value obtained byconverting a time from the start of reading by the reading portion 244 bto the completion of the conversion processing of the A/D converter 246into the number of frames. The memory 247 includes a random accessmemory (RAM), a read only memory (ROM), a flash memory, and the like.

The operating portion 22 includes a bending knob 221 that bends thebending portion 25 in a vertical direction and a horizontal direction, atreatment tool insertion portion 222 that inserts a treatment tool suchas biopsy forceps, an electric scalpel, and an inspection probe into thebody cavity of the subject, and a plurality of switches 223 that areoperation input/output portions inputting operation instruction signalsof peripheral devices such as an air supply unit, a water supply unit,and screen display control, in addition to the processing device 3. Thetreatment tool inserted from the treatment tool insertion portion 222comes out from an aperture portion (not illustrated) via a treatmenttool channel (not illustrated) of the distal end portion 24.

The universal cord 23 incorporates at least the light guide 241 and anassembly cable 248 including a plurality of signal lines. The assemblycable 248 includes a signal line for transmitting an imaging signal, asignal line for transmitting a drive signal for driving the image sensor244, and a signal line for transmitting and receiving informationincluding identification information regarding the endoscope 2 (imagesensor 244) and the like. Note that, in the present embodiment,transmitting the electric signal using a signal line is described, butan optical signal may be transmitted, or a signal may be transmittedbetween the endoscope 2 and the processing device 3 by wirelesscommunication.

Next, a configuration of the processing device 3 will be described. Theprocessing device 3 includes the illumination portion 3 a and aprocessor portion 3 b. The processor portion 3 b includes an imageprocessing portion 31, a first communication portion 32, a total delaytime detection portion 33, a synchronization signal generation portion34, a second communication portion 35, an input/output portion 36, acontrol portion 37, and a storage portion 38. The processor portion 3 bcorresponds to an image processing device.

First, a configuration of the illumination portion 3 a will bedescribed. The illumination portion 3 a includes a light source portion310 and an illumination control portion 320.

The light source portion 310 is configured using a light source thatemits illumination light, a plurality of lenses, a filter that passeslight in a predetermined wavelength band, and the like, and emitsillumination light of the light in the predetermined wavelength band.The light source portion 310 has a light source 310 a, a light sourcedriver 310 b, a rotating filter 310 c, a drive portion 310 d, and adriver 310 e.

The light source portion 310 is configured using a white LED, one ormore lenses, and the like, and emits white light to the rotating filter310 c under the control of the light source driver 310 b. The whitelight generated by the light source 310 a is emitted from the distal endof the distal end portion 24 to the subject via the rotating filter 310c and the light guide 241.

The light source driver 310 b supplies a current to the light source 310a under the control of the illumination control portion 320 to cause thelight source 310 a to emit white light.

The rotating filter 310 c is disposed on an optical path of the whitelight emitted from the light source 310 a, and rotates to transmit onlylight in a predetermined wavelength band among the white light emittedfrom the light source 310 a. Specifically, the rotating filter 310 c hasa red filter 311, a green filter 312, and a blue filter 313 thattransmit light having wavelength bands of red light (R), green light(G), and blue light (B), respectively. The rotating filter 310 c rotatesto sequentially transmit light in red, green, and blue wavelength bands(for example, red: 600 nm to 700 nm, green: 500 nm to 600 nm, and blue:400 nm to 500 nm). As a result, any one of red light (R illumination),green light (G illumination), and blue light (B illumination) can besequentially emitted to the endoscope 2 using the white light (Willumination) emitted from the light source 310 a (frame sequentialmethod).

The drive portion 310 d includes a stepping motor, a DC motor, or thelike, and rotates the rotating filter 310 c.

The driver 310 e supplies a predetermined current to the drive portion310 d under the control of the illumination control portion 320.

The illumination control portion 320 controls an amount of current to besupplied to the light source 310 a, on the basis of the control signalfrom the control portion 37. Further, the illumination control portion320 drives the drive portion 310 d via the light source driver 310 bunder the control of the control portion 37 to rotate the rotatingfilter 310 c.

Note that the light source 310 a may include a red LED, a green LED, anda blue LED, and the light source driver 310 b may supply a current toeach LED to sequentially emit red light, green light, or blue light. Inaddition, light may be simultaneously emitted from a white LED, a redLED, a green LED, and a blue LED, or an image may be acquired byirradiating the subject with white light using a laser, a discharge lampsuch as a xenon lamp, or the like.

The image processing portion 31 receives an imaging signal of theillumination light of each color imaged by the image sensor 244 from theendoscope 2. The image processing portion 31 performs predeterminedimage processing on the imaging signal received from the endoscope 2,generates an imaging signal for display, and outputs the imaging signalto the display device 4. The image processing portion 31 includes one ora combination of a general-purpose processor such as a centralprocessing portion (CPU) and a dedicated processor such as variousarithmetic circuits executing specific functions such as an applicationspecific integrated circuit (ASIC) and a field programmable gate array(FPGA) to be a programmable logic device capable of rewriting processingcontents.

FIG. 3 is a diagram illustrating a configuration of the image processingportion included in the endoscope system according to the firstembodiment of the disclosure. The image processing portion 31 has a D/Aconverter 501, a dimming detection portion 502, an automatic gaincontrol (AGC) portion 503, a noise removal portion 504, a white balance(WB) processing portion 505, a freeze processing portion 506, a gammaprocessing portion 507, a color matrix processing portion 508, a scalingprocessing portion 509, an emphasis processing portion 510, an A/Dconverter 511, a first frame memory 521, a second frame memory 522, anda third frame memory 523.

The D/A converter 501 converts a digital imaging signal input from theendoscope 2 into an analog signal. Hereinafter, the image processingportion 31 performs various types of processing on data (image data)related to image display among the data included in the imaging signal.

The dimming detection portion 502 detects a brightness levelcorresponding to each image, on the basis of RGB image informationincluded in the image data input from the D/A converter 501. The dimmingdetection portion 502 sets light emission conditions such as an amountof light generated by the illumination portion 3 a and light emissiontiming, on the basis of the detected brightness level.

The AGC portion 503 performs processing of adjusting an amplificationfactor (gain) of a signal value on the image data to maintain a constantoutput level. The AGC portion 503 performs gain adjustment according tothe light emission conditions set by the dimming detection portion 502.

The noise removal portion 504 performs noise reduction processing on theimage data input from the AGC portion 503. The noise removal portion 504acquires image data of a previous frame with reference to, for example,the first frame memory 521, and removes noise using the acquired imagedata and image data to be subjected to noise removal processing. A knownmethod can be used for the noise reduction processing.

The WB processing portion 505 performs processing of correcting whitebalance on the image data after the noise removal processing. A knownmethod can be used for the white balance correction processing.

When the input of the freeze instruction signal is received by pressingthe switch 223, the freeze processing portion 506 refers to the secondframe memory 522 and selects image data to be frozen and displayed onthe display device 4. The freeze processing portion 506 outputs theselected image data to the gamma processing portion 507. The freezeprocessing portion 506 selects image data with small blurring from theimage data stored in the second frame memory 522, for example.

On the other hand, when the input of the freeze instruction signal isnot received, the freeze processing portion 506 outputs predeterminedimage data in the second frame memory 522, for example, image data withthe latest acquisition (imaging) time to the gamma processing portion507. Here, when the latest image data is acquired, the freeze processingportion 506 may acquire the image data from the WB processing portion505.

After selecting the image data to be frozen, the freeze processingportion 506 performs only processing of outputting the image data inputfrom the WB processing portion 505 to the second frame memory 522 duringa still image display period by freeze processing. After the freezeprocessing is canceled, the freeze processing portion 506 selects thelatest image data from the second frame memory 522, including the imagedata newly input from the WB processing portion 505. For this reason, inthe image displayed on the display device 4 after the freeze processing,an image in which the latest image data in the still image displayperiod by the freeze processing is omitted and which is spaced apart intime series from that before the freeze processing. Due to the omissionin the image data of a predetermined number of frames, a change in thesubject image may be larger in the images displayed before and after thefreeze processing as compared with a case where images adjacent in timeseries are displayed as moving images.

The gamma processing portion 507 performs gamma correction processing onthe image data input from the freeze processing portion 506. The gammaprocessing portion 507 performs, on the image data, gradation correctionfor increasing the brightness of a dark portion having a small luminanceby using a preset y value. A known method can be used for the gammacorrection processing.

The color matrix processing portion 508 performs color correction forimproving color reproducibility on the image data input from the gammaprocessing portion 507. The color matrix processing portion 508uniformly manages different colors among devices for the image data. Thecolor matrix processing portion 508 uniformly manages colors between theendoscope 2, the processing device 3, and the display device 4, forexample. The color matrix processing portion 508 is configured using acolor management system (CMS). A known method can be used for the colormatrix processing.

The scaling processing portion 509 performs processing of changing thesize of the image data, according to a preset enlargement ratio or apreset reduction ratio, or an enlargement ratio or a reduction ratioinput via the input/output portion 36. Note that the processing of thescaling processing portion 509 may not be enlarged or reduced bysetting. The propriety of the zoom processing may be switched accordingto, for example, information received by the input/output portion 36.

The emphasis processing portion 510 performs contour emphasis processingon the image data after the scaling processing. By the emphasisprocessing of the emphasis processing portion 510, image data in which acontour is more clearly expressed is generated. A known method can beused for the contour emphasis processing.

The A/D converter 511 converts the analog image data output through theemphasis processing portion 510 into a digital signal. The A/D converter511 outputs the image data after the digital conversion to the displaydevice 4 as an imaging signal for image display.

The first frame memory 521, the second frame memory 522, and the thirdframe memory 523 store the image data generated by the connectedportions for the set frames. In the first embodiment, each frame memorystores image data of several frames. When new image data is input, eachframe memory overwrites the oldest image data with the new image dataamong the currently stored image data, thereby sequentially updating andstoring image data for several frames in order from the latestacquisition time. The first frame memory 521, the second frame memory522, and the third frame memory 523 are configured using a random accessmemory (RAM), for example, a video RAM (VRAM).

When the endoscope 2 is connected, the first communication portion 32acquires the processing time stored in the memory 247 of the endoscope2. The first communication portion 32 is configured using ageneral-purpose processor such as a CPU or a dedicated processor such asvarious arithmetic circuits executing specific functions such as anASIC.

The total delay time detection portion 33 detects a delay time requiredfor signal processing from generation of an imaging signal for one frameto display of image data on the display device 4 in the endoscope 2, theprocessing device 3, and the display device 4. Specifically, the totaldelay time detection portion 33 calculates the sum of the delay time ofthe endoscope 2, the delay time of the processing device 3, and thedelay time of the display device 4. A value calculated by the totaldelay time detection portion 33 in the first embodiment is a valueobtained by converting the total delay time into the number of frames.The total delay time detection portion 33 is configured using ageneral-purpose processor such as a CPU or a dedicated processor such asvarious arithmetic circuits executing specific functions such as anASIC.

The synchronization signal generation portion 34 generates a clocksignal (synchronization signal) serving as a reference of the operationof the processing device 3, and outputs the generated synchronizationsignal to the illumination portion 3 a, the image processing portion 31,the control portion 37, and the endoscope 2. Here, the synchronizationsignal generated by the synchronization signal generation portion 34includes a horizontal synchronization signal and a verticalsynchronization signal.

Therefore, the illumination portion 3 a, the image processing portion31, the control portion 37, and the endoscope 2 operate insynchronization with each other according to the generatedsynchronization signal.

The synchronization signal generation portion 34 is configured using ageneral-purpose processor such as a CPU or a dedicated processor such asvarious arithmetic circuits executing specific functions such as anASIC.

When the display device 4 is connected, the second communication portion35 acquires a processing time required for the signal processing in thedisplay device 4, which is stored in a memory 44 of the display device4. The second communication portion 35 is configured using ageneral-purpose processor such as a CPU or a dedicated processor such asvarious arithmetic circuits executing specific functions such as anASIC.

The input/output portion 36 is realized by using a keyboard, a mouse, aswitch, and a touch panel, and receives inputs of various signals suchas an operation instruction signal that instructs an operation of theendoscope system 1. Further, the input/output portion 36 is realized byusing at least one of a speaker and a light source, and outputs sound orlight. Note that the input/output portion 36 may include a switchprovided in the operating portion 22 or a portable terminal such as anexternal tablet computer.

The control portion 37 performs drive control of each componentincluding the image sensor 244 and the illumination portion 3 a,input/output control of information with respect to each component, andthe like. The control portion 37 refers to control information data (forexample, reading timing or the like) for imaging control stored in thestorage portion 38, and transmits the control information data as adrive signal to the image sensor 244 via a predetermined signal lineincluded in the assembly cable 248. The control portion 37 is configuredusing a general-purpose processor such as a CPU or a dedicated processorsuch as various arithmetic circuits executing specific functions such asan ASIC.

The storage portion 38 stores various programs for operating theendoscope system 1, data including various parameters and the likenecessary for the operation of the endoscope system 1, and a delay timerequired for signal processing in the processing device 3. Further, thestorage portion 38 stores identification information of the processingdevice 3. Here, the identification information includes identificationinformation (ID), a model year, specification information, and the likeof the processing device 3. The processing time corresponds to a delaytime of data transmission caused by signal processing in the processingdevice 3, and is a value obtained by converting a time from when imagedata is input from the endoscope 2 to when the image data is output tothe display device 4 into the number of frames. Further, the storageportion 38 has a signal processing information storage portion 381 thatstores signal processing conditions for controlling signal processing inthe entire system including the endoscope 2 and the display device 4.The signal processing information storage portion 381 stores, forexample, one or more thresholds set for the total delay time detected bythe total delay time detection portion 33.

Further, the storage portion 38 stores various programs including animage acquisition processing program for executing an image acquisitionprocessing method of the processing device 3. The various programs canbe recorded on a computer-readable recording medium such as a hard disk,a flash memory, a CD-ROM, a DVD-ROM, or a flexible disk and widelydistributed. Note that the above-described various programs can also beacquired by being downloaded via a communication network. Here, thecommunication network is realized by, for example, an existing publicline network, a local area network (LAN), a wide area network (WAN), orthe like, and may be a wired or wireless network.

The storage portion 38 having the above configuration is realized byusing a read only memory (ROM) in which various programs and the likeare previously installed, and a RAM, a hard disk, and the like thatstore calculation parameters, data, and the like of each processing.

The display device 4 displays a display image corresponding to the imagedata received from the processing device 3 (image processing portion 31)via a video cable. The display device 4 includes a display processingportion 41, a display portion 42, and the memory 44.

The display processing portion 41 performs predetermined processing, forexample, synchronization processing, format conversion, or the like, onthe image data received from the processing device 3, and outputs theimage data to the display portion 42. The synchronization processing isprocessing of synchronizing each of R image data based on image datagenerated by the image sensor 244 when the light source portion 310emits the R illumination light, G image data based on image datagenerated by the image sensor 244 when the light source portion 310emits the G illumination light, and B image data based on image datagenerated by the image sensor 244 when the light source portion 310emits the B illumination light.

The display portion 42 is configured using a monitor such as liquidcrystal or organic electro luminescence (EL).

A notification portion 43 is realized by using at least one of a speakerand a light source, and outputs sound or light. The notification portion43 emits sound and light according to the degree of display delay.

In the memory 44, a processing time for storing an execution program anda control program to execute various operations by the display device 4,or a processing time required for signal processing in the displaydevice 4 corresponds to a delay time of data transmission caused by thesignal processing, and is a value obtained by converting a time fromwhen image data is input from the processing device 3 to when the imagedata is displayed on the display portion 42 into the number of frames.The memory 44 includes a RAM, a ROM, a flash memory, and the like.

Here, an image captured by the endoscope 2 and an image displayed on thedisplay device 4 in a case where signal processing is delayed in theendoscope system 1 will be described with reference to FIG. 4. FIG. 4 isa diagram illustrating an image displayed by image processing performedby the endoscope system according to the first embodiment of thedisclosure, and is a timing chart illustrating imaging timing anddisplay timing. In FIG. 4, a horizontal axis represents the lapse oftime. (a) of FIG. 4 illustrates an image captured by the image sensor244 of the endoscope 2. (b) of FIG. 4 illustrates an image displayed ondisplay device 4 without delay. (c) of FIG. 4 illustrates an imagedisplayed on the display device 4 when the display time is delayed byone frame. (d) of FIG. 4 illustrates an image displayed on the displaydevice 4 when the display time is delayed by two frames.

In the endoscope 2, image data for one frame is sequentially captured(see (a) of FIG. 4). Specifically, images are sequentially captured inthe order of images F₁₁, F₁₂, F₁₃, . . . , F₁₆, F₁₇, . . . . A subject Sis captured in the images F₁₂, F₁₃, . . . , and F₁₆.

In a case where the image data captured by the endoscope 2 is displayedon the display device 4 through the above-described processing, if nodelay occurs in each processing, the image data of each frame issequentially displayed (see (b) of FIG. 4). Specifically, images F₂₁,F₂₂, F₂₃, . . . , F₂₆, and F₂₇ corresponding to the images F₁₁, F₁₂,F₁₃, . . . , F₁₆, and F₁₇, are sequentially displayed. Each image(images F₂₁, F₂₂, F₂₃, . . . , F₂₆, F₂₇, . . . ) is displayed on thedisplay device 4 at the same timing as the imaging timing in theendoscope 2.

On the other hand, when a delay occurs during a period until an image iscaptured by the endoscope 2 and is then displayed on the display device4 through the above-described processing, the image data of each frameis sequentially displayed with a delay corresponding to the delay (see(b) and (c) of FIG. 4). For example, when a delay of one frame occurs,images F₃₁, F₃₂, F₃₃, . . . , and F₃₆ corresponding to the images F₁₁,F₁₂, F₁₃, . . . , and F₁₆ are displayed on the display device 4 afterbeing delayed by one frame with respect to the imaging timing in theendoscope 2 (see (b) of FIG. 4). Further, when a delay of two framesoccurs, images F₄₁, F₄₂, F₄₃, F₄₄, and F₄₅ corresponding to the imagesF₁₁, F₁₂, F₁₃, F₁₄, and F₁₅ are displayed on the display device 4 afterbeing delayed by two frames with respect to the imaging timing in theendoscope 2 (see (c) of FIG. 4).

When the display is delayed in units of frames, for example, when thedisplay is delayed by two frames, while the distal end portion 24 of theendoscope 2 is at a position where the image F₁₃ is captured, the imageF₄₁ according to the image F₁₁ is displayed on the display portion 42 ofthe display device 4. In this case, a deviation occurs between theactual position of the endoscope 2 and the position of the endoscope 2which is recognized by the operator and is identified from the imagedisplayed on the display portion 42. If the number of delayed framesincreases, the deviation also increases accordingly.

In the first embodiment, when the above-described display delay occurs,the notification portion 43 is caused to execute notification processingaccording to the degree of delay.

Next, delay detection processing performed by the endoscope system 1will be described. FIG. 5 is a flowchart illustrating the delaydetection processing performed by the endoscope system according to thefirst embodiment of the disclosure.

When the endoscope 2 and the display device 4 are connected after poweris supplied, the processing device 3 acquires processing information(delay time) of each device from the memories 247 and 44 (step S101).

In step S102 following step S101, the total delay time detection portion33 refers to the storage portion 38 to acquire the processing time(delay time) in the processing device 3, and calculates the sum of thedelay times of the respective devices.

In step S103 following step S102, the control portion 37 compares thesum of the delay times of the respective devices with a threshold, anddetermines whether or not a delay to be notified can occur. Here, thethreshold is preset and stored in the storage portion 38, and is a valuedetermined by the number of frames. As the threshold, for example, threeor more frames are set. Note that the threshold may be set (changed) inaccordance with the information received by the input/output portion 36and associated with the threshold set by the user such as the operator.

Here, when the control portion 37 determines that the sum of the delaytimes of the respective devices is equal to or more than the threshold,that is, a delay to be notified can occur on the basis of the sum (stepS103: Yes), the control portion 37 proceeds to step S104. On the otherhand, when the control portion 37 determines that the sum of the delaytimes of the respective devices is less than the threshold, that is,that no delay to be notified occurs on the basis of the sum (step S103:No), the control portion 37 ends the delay detection processing.

In step 3104, the control portion 37 causes the notification portion 43of the display device 4 to notify that a delay occurs in the displayedimage. The notification portion 43 outputs sound or light to notify theoperator that a deviation occurs between the actual position of theendoscope 2 and the position of the endoscope 2 recognized by theoperator.

At this time, the intensity of the notification may be changed stepwise.For example, a plurality of thresholds for notification may be set, andthe loudness of sound and the intensity of light may be changed when thetotal delay time (the number of frames) increases.

In addition to the notification portion 43, the input/output portion 36may execute the notification processing.

In the first embodiment described above, the delay time generated by theimage generation processing in each of the endoscope 2, the processingdevice 3, and the display device 4 is detected, and the notificationportion 43 notifies the delay time according to the detection result.The operator can determine that the position of the distal end portion24 of the endoscope 2 recognized by the operator from the display imageis deviated from the actual position by the notification processing ofthe notification portion 43. According to the first embodiment, it ispossible to suppress the deviation between the actual position of theendoscope 2 and the position of the endoscope 2 recognized by theoperator observing the display image.

Note that, in the first embodiment, the example in which thenotification portion 43 performs the notification processing accordingto the degree of delay has been described, but information indicatingthat a deviation occurs between the actual position of the endoscope 2and the position of the endoscope 2 recognized by the operator may bedisplayed on the display portion 42.

Second Embodiment

Next, a second embodiment of the disclosure will be described withreference to FIG. 6. FIG. 6 is a flowchart illustrating delay detectionprocessing performed by an endoscope system according to the secondembodiment of the disclosure. Note that, since a configuration of theendoscope system according to the second embodiment is the same as thatof the above-described endoscope system, the description thereof will beomitted. Hereinafter, processing different from that of the firstembodiment will be described.

When an endoscope 2 and a display device 4 are connected after power issupplied, a processing device 3 acquires processing information (delaytime) of each device from memories 247 and 44 (step S201).

In step S202 following step S201, a total delay time detection portion33 refers to a storage portion 38 to acquire the processing time (delaytime) in the processing device 3, and calculates the sum of the delaytimes of the respective devices.

In step S203 following step S202, a control portion 37 compares the sumof the delay times of the respective devices with a threshold, anddetermines whether or not a delay to be notified can occur. Here, thethreshold is the same as that in the first embodiment.

Here, when the control portion 37 determines that the sum of the delaytimes of the respective devices is equal to or more than the threshold,that is, a delay to be notified can occur on the basis of the sum (stepS203: Yes), the control portion 37 proceeds to step S204. On the otherhand, when the control portion 37 determines that the sum of the delaytimes of the respective devices is less than the threshold, that is, nodelay to be notified occurs on the basis of the sum (step S203: No), thecontrol portion 37 proceeds to step S205.

In step S204, the control portion 37 causes an image processing portion31 to perform intermittent processing. In the intermittent processing, apart of the processing is thinned out according to the total delay time(the number of frames). Examples of processing of a thinning-out targetinclude freeze processing, emphasis processing, and the like. When thetotal delay time is longer (the number of frames is larger), the numberof thinning-out processing also increases. At this time, thethinning-out priority order of each processing is preset and stored inthe storage portion 38. The control portion 37 determines thinning-outprocessing with reference to the total delay time and the priorityorder.

Note that, when processing of generating an all-in-focus image bycombining images at different focal points, processing of generating athree-dimensional image, processing corresponding to 4K resolution,color shift correction processing of a moving image, and the like areperformed, these processing are processing to be thinned out.

In addition, the delay time may be reduced by imposing restrictions onthe monitor output format, or when Display Port, HDMI, or DVI is used, achange to SDI or the like may be prompted. By changing to the SDI, thedelay time can be reduced as compared with Display Port, HDMI, and DVI.

On the other hand, in step S205, the control portion 37 causes the imageprocessing portion 31 to perform normal processing. In the normalprocessing, each portion (see FIG. 3) of the above-described imageprocessing portion 31 is caused to execute processing.

In the second embodiment described above, the delay time generated bythe image generation processing in each of the endoscope 2, theprocessing device 3, and the display device 4 is detected, and contentsof the image processing are changed according to the detection result.According to the second embodiment, the delay is eliminated by thinningout a part of the image processing, and as a result, it is possible tosuppress a deviation between an actual position of the endoscope 2 and aposition of the endoscope 2 recognized by an operator observing adisplay image.

Third Embodiment

Next, a third embodiment of the disclosure will be described withreference to FIG. 7. FIG. 7 is a flowchart illustrating delay detectionprocessing performed by an endoscope system according to the thirdembodiment of the disclosure. Note that, since a configuration of theendoscope system according to the third embodiment is the same as thatof the above-described endoscope system, the description thereof will beomitted. Hereinafter, processing different from those in the first andsecond embodiments will be described. In the third embodiment,notification processing or image processing change processing isexecuted on the basis of a delay time.

When an endoscope 2 and a display device 4 are connected after power issupplied, a processing device 3 acquires processing information (delaytime) of each device from memories 247 and 44 (step S301).

In step S302 following step S301, a total delay time detection portion33 refers to a storage portion 38 to acquire the processing time (delaytime) in the processing device 3, and calculates the sum of the delaytimes of the respective devices.

In step S303 following step S302, a control portion 37 compares the sumof the delay times of the respective devices with a first threshold, anddetermines whether or not a delay to be notified or subjected to aprocessing change can occur. Here, the first threshold is a numericalvalue determined by the number of frames, similarly to the firstembodiment and the like.

Here, when the control portion 37 determines that the sum of the delaytimes of the respective devices is equal to or more than the firstthreshold, that is, a delay to be notified can occur on the basis of thesum (step S303: Yes), the control portion 37 proceeds to step S304. Onthe other hand, when the control portion 37 determines that the sum ofthe delay times of the respective devices is less than the firstthreshold, that is, that no delay to be notified occurs on the basis ofthe sum (step S303: No), the control portion 37 ends the delay detectionprocessing.

In step S304, the control portion 37 compares the sum of the delay timesof the respective devices with a second threshold, and determineswhether or not a delay to be notified can occur. Here, the secondthreshold is a numerical value determined by the number of frames and isa numerical value larger than the first threshold.

Here, when the control portion 37 determines that the sum of the delaytimes of the respective devices is equal to or more than the secondthreshold, that is, a delay to be notified can occur on the basis of thesum (step S304: Yes), the control portion 37 proceeds to step S305. Onthe other hand, when the control portion 37 determines that the sum ofthe delay times of the respective devices is less than the secondthreshold, that is, no delay to be notified occurs on the basis of thesum (step S304: No), the control portion 37 proceeds to step S306.

In step S305, the control portion 37 causes a notification portion 43 ofthe display device 4 to notify that a delay occurs in the displayedimage, similarly to step S104 of the first embodiment. At this time, thecontrol portion 37 causes an image processing portion 31 to performnormal processing of causing each portion (see FIG. 3) of theabove-described image processing portion 31 to execute processing.

In step S306, the control portion 37 causes the image processing portion31 to perform intermittent processing, similarly to step S204 of thesecond embodiment.

In the third embodiment described above, the delay time generated by theimage generation processing in each of the endoscope 2, the processingdevice 3, and the display device 4 is detected, and whether thenotification portion 43 performs notification or contents of the imageprocessing are changed is selected according to the detection result.According to the third embodiment, it is possible to suppress adeviation between an actual position of the endoscope 2 and a positionof the endoscope 2 recognized by an operator observing a display imageby the notification of the delay or the elimination of the delay.

Fourth Embodiment

Next, a fourth embodiment of the disclosure will be described withreference to FIGS. 8 and 9. FIG. 8 is a block diagram illustrating aschematic configuration of an endoscope system according to the fourthembodiment of the disclosure.

An endoscope system 1A illustrated in FIG. 8 includes an endoscope 2that captures an image of a subject by inserting a distal end portioninto the subject, a processing device 3A that has an illuminationportion 3 a generating illumination light emitted from the distal end ofthe endoscope 2, performs predetermined signal processing on an imagingsignal captured by the endoscope 2, and generally controls the entireoperation of the endoscope system 1A, and a display device 4A thatdisplays an in-vivo image generated by the signal processing of theprocessing device 3A. The endoscope system 1A according to the fourthembodiment has the same configuration except that a processing device 3of an endoscope system 1 described above is changed to the processingdevice 3A and a display device 4 is changed to the display device 4A.Hereinafter, a configuration different from that of the first embodimentwill be described.

The display device 4A displays a display image corresponding to imagedata received from the processing device 3A (image processing portion31) via a video cable. The display device 4A includes a displayprocessing portion 41, a display portion 42, and a notification portion43. The display device 4A according to the fourth embodiment has aconfiguration not including a memory 44 described above. Therefore, theprocessing device 3A cannot acquire a delay time from the display device4A. A configuration including the memory 44 may be adopted, but in thefourth embodiment, the delay time is not stored.

The processing device 3A includes an illumination portion 3 a, an imageprocessing portion 31, a first communication portion 32, a total delaytime detection portion 33, a synchronization signal generation portion34, an input/output portion 36, a control portion 37, a storage portion38, and a delay time acquisition portion 39. The processing device 3A isconfigured to include the delay time acquisition portion 39 instead ofthe second communication portion 35 in the processing device 3 describedabove. Hereinafter, the delay time acquisition portion 39 will bedescribed.

The delay time acquisition portion 39 includes a test pattern outputportion 391 and a phase comparison portion 392. The delay timeacquisition portion 39 is configured using a general-purpose processorsuch as a CPU or a dedicated processor such as various arithmeticcircuits executing specific functions such as an ASIC.

The test pattern output portion 391 outputs image data of apredetermined pattern to the display device 4A.

The phase comparison portion 392 compares phases of the image of thepattern output by the test pattern output portion 391 and the pattern ofthe display image processed in the display device 4A. Here, the phaserefers to an appearance pattern of brightness of sequentially outputimages. The phase comparison portion 392 obtains a phase differencebetween the image of the pattern output by the test pattern outputportion 391 and the pattern of the display image processed in thedisplay device 4A as a delay time.

FIG. 9 is a diagram illustrating delay time acquisition processingperformed by the endoscope system according to the fourth embodiment ofthe disclosure, and is a timing chart illustrating pattern output timingand display timing. In FIG. 9, a horizontal axis represents the lapse oftime. (a) of FIG. 9 illustrates an image corresponding to the patternoutput from the test pattern output portion 391. (b) of FIG. 9illustrates the pattern image processed in the display device 4A andinput to the phase comparison portion. In the display device 4A, when atest pattern is input from the processing device 3A, processing fordisplay is performed by the display processing portion 41. The imagedata processed by the display processing portion 41 is input to thephase comparison portion 392. This image data is recognized as an imageof the same frame as a frame displayed by the display portion 42.

When the test pattern output portion 391 receives an executioninstruction of the delay time acquisition processing of the displaydevice 4A from the control portion 37, the test pattern output portion391 sequentially outputs test patterns to the display device 4A (see (a)of FIG. 9). Specifically, the test patterns are sequentially output inthe order of pattern images F₅₁, F₅₂, F₅₃, . . . , F₅₉, F₆₀, . . . , andF₆₅. In the pattern images F₅₁, F₅₂, F₅₃, . . . , F₅₉, F₆₀, . . . , andF₆₅, test patterns having different brightness are output at presetintervals. In (a) of FIG. 9, the pattern images F₅₅, F₆₀, and F₆₅correspond to the test patterns having different brightness.

The test patterns processed in the display device 4A are sequentiallyinput from the display processing portion 41 to the phase comparisonportion 392 (see (b) of FIG. 9). Specifically, the test patterns aresequentially output in the order of pattern images F₇₁, F₇₂, F₇₃, . . ., F₇₉, F₈₀, . . . , and F₈₃. Even in the pattern images F₇₁, F₇₂, F₇₃, .. . , F₇₉, F₈₀, . . . , and F₉₃, test patterns having differentbrightness appear. In (b) of FIG. 9, the pattern images F₇₅ and F₈₀correspond to the test patterns having different brightness.

As can be seen from (a) and (b) of FIG. 9, a delay of two frames occursdue to the processing of the display device 4A. The phase comparisonportion 392 calculates the delay time by comparing the phases from thebrightness between the test pattern output from the test pattern outputportion 391 to the display device 4A and the test pattern returned fromthe display device 4A.

The delay detection processing according to the fourth embodiment isperformed according to the processing of the first or second embodimentdescribed above. At this time, the total delay time detection portion 33acquires the delay time calculated by the phase comparison portion 392(steps S101 and S201).

In the fourth embodiment described above, even when the delay time ofthe display device 4A is unknown, the test patterns are output, thephases are compared, and the delay time is acquired. According to thefourth embodiment, even in a device with an unknown delay time, it ispossible to acquire the delay time of the device and perform processingaccording to the delay time.

Note that, in the above-described first to fourth embodiments, it hasbeen described that the illumination portion 3 a is configuredseparately from the endoscope 2, but for example, a configuration inwhich a light source device is provided in the endoscope 2, such asproviding a semiconductor light source at the distal end of theendoscope 2, may be adopted. Further, the functions of the processingdevices 3 and 3A may be given to the endoscope 2.

Further, in the above-described first to fourth embodiments, it has beendescribed that the illumination portion 3 a is integrated with theprocessing devices 3 and 3A, but the illumination portion 3 a and theprocessing device 3 may be separated, and for example, the light sourceportion 310 and the illumination control portion 320 may be providedoutside the processing devices 3 and 3A.

Further, in the above-described first to fourth embodiments, a part ofthe functions of the processor portion 3 b may be executed by theendoscope 2 or the display device 4. For example, the total delay timedetection portion 33 may be provided in the endoscope 2 or the displaydevice 4 to detect the delay time in devices other than the processingdevices 3 and 3A.

Further, in the above-described first to fourth embodiments, theendoscope system according to the disclosure has been described as theendoscope system 1 using the flexible endoscope 2 in which theobservation target is a living tissue or the like in the subject.However, the disclosure can also be applied to an endoscope system usinga rigid endoscope, an industrial endoscope for observing characteristicsof a material, a capsule endoscope, a fiber scope, or an opticalendoscope such as an optical visual tube in which a camera head isconnected to an eyepiece.

According to the disclosure, it is possible to suppress a deviationbetween an actual position of an endoscope and a position of theendoscope recognized by an operator viewing a display image.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the disclosure in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

What is claimed is:
 1. An endoscope system comprising: an endoscopeconfigured to capture an image of a subject, generate an imaging signal,and output the generated imaging signal; an image processing deviceconfigured to perform image processing on the imaging signal input fromthe endoscope; a display configured to display the image of the subjectbased on the imaging signal subjected to the image processing by theimage processing device; and a first processor comprising hardware, thefirst processor being provided in any one of the endoscope, the imageprocessing device, and the display, the first processor being configuredto calculate a sum of a first processing time from when the endoscopegenerates the imaging signal to when the endoscope outputs the imagingsignal, a second processing time from when the image processing devicereceives the imaging signal to when the image processing device outputsthe imaging signal to the display, and a third processing time from whenthe display receives the imaging signal to when the display displays theimage based on the imaging signal.
 2. The endoscope system according toclaim 1, wherein, the endoscope includes an image sensor configured togenerate the imaging signal by photoelectrically converting receivedlight and reading a photoelectrically converted electric signal, and asecond processor comprising hardware, the second processor beingconfigured to perform signal processing on the imaging signal generatedby the image sensor, the image processing device includes a thirdprocessor comprising hardware, the third processor being configured toperform a plurality of types of processing on the imaging signal inputfrom the endoscope, the display includes a fourth processor comprisinghardware, the fourth processor being configured to perform processingfor image display based on the imaging signal input from the imageprocessing device, and the first processor is further configured tocalculate a sum of a first processing time required for processing bythe image sensor and the second processor, a second processing timerequired for processing by the third processor, and a third processingtime required for processing by the fourth processor.
 3. The endoscopesystem according to claim 1, further comprising: a notification portionconfigured to notify that a deviation occurs between a current positionof the endoscope and a position of the endoscope recognized from theimage of the subject displayed on the display when the sum calculated bythe first processor is equal to or more than a predetermined threshold.4. The endoscope system according to claim 2, wherein the imageprocessing device further includes a controller configured to cause thethird processor to perform intermittent processing in which a part ofthe processing to be performed by the third processor is thinned outwhen the sum calculated by the first processor is equal to or more thana predetermined threshold.
 5. The endoscope system according to claim 2,wherein, the image processing device further includes a fifth processorcomprising hardware, the fourth processor being configured to outputfirst reference image data that is preset, to the display, acquiresecond reference image data that is obtained by performing theprocessing on the first reference image by the fourth processor of thedisplay, and compare a phase of the output first reference image datawith a phase of the acquired second reference image data to acquire thethird processing time required for processing by the fourth processor,and the first processor is further configured to calculate a sum of thefirst and second processing times and the third processing time acquiredby the fifth processor.
 6. The endoscope system according to claim 1,wherein, the first processor is further configured to calculate a sum ofthe first processing time stored in the endoscope, the second processingtime stored in the image processing device, and the third processingtime stored in the display.
 7. An image processing device that isconnected to an endoscope and a display, the endoscope being configuredto capture an image of a subject, generate an imaging signal, and outputthe generated imaging signal, the display being configured to displaythe image of the subject, the image processing device comprising: aprocessor comprising hardware, the processor being configured to performimage processing on the imaging signal input from the endoscope,calculate a sum of a first processing time from when the endoscopegenerates the imaging signal to when the endoscope outputs the imagingsignal, a second processing time from when the image processing devicereceives the imaging signal to when the image processing device outputsthe imaging signal to the display, and a third processing time from whenthe display receives the imaging signal to when the display displays theimage based on the imaging signal.
 8. A total processing time detectionmethod comprising: acquiring a first processing time from when anendoscope generates an imaging signal to when the endoscope outputs theimaging signal; acquiring a second processing time from when an imageprocessing device receives the imaging signal to when the imageprocessing device outputs the imaging signal to a display; acquiring athird processing time from when the display receives the imaging signalto when the display displays the image based on the imaging signal; andcalculating a sum of the first processing time, the second processingtime, and the third processing time.
 9. A processing device comprising:an input portion configured to receive an imaging signal from anendoscope; an output portion configured to output a display image to adisplay; and a processor comprising hardware, the processor beingconfigured to acquire a processing time from when the input portionreceives the imaging signal to when the output portion outputs thedisplay image, and perform notification processing when the processingtime is equal to or more than a predetermined threshold.
 10. Aprocessing device comprising: an image processing circuit configured toexecute first image processing and second image processing on an imagingsignal, the second image processing partially different from the firstimage processing, a processing time of the second image processing beingshorter than a processing time of the first image processing; and acontrol circuit configured to acquire a processing time from when animage sensor outputs an imaging signal to when a display displays animage, determine whether the processing time is equal to or more than apredetermined threshold, and perform switching from the first imageprocessing to the second image processing when the processing time isequal to or more than the predetermined threshold.
 11. A processingdevice comprising: a processor comprising hardware, the processor beingconfigured to acquire a time from when the image sensor outputs animaging signal to when a display displays an image generated based onthe imaging signal, and perform notification when the time is equal toor more than a predetermined threshold.
 12. A processing devicecomprising: a processor comprising hardware, the processor beingconfigured to perform image processing on an imaging signal to generatean image, acquire a time from when an image sensor outputs the imagingsignal to when a display displays the image, and switch the imageprocessing from first image processing to second image processing whenthe time is equal to or more than a predetermined threshold, the secondimage processing being image processing in which a part of the firstimage processing is thinned out.